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Abstract 

For macroscopic systems, the second law of thermodynamics estab- 
lishes an inequality between the amount of work performed on a system 
in contact with a thermal reservoir, and the change in its free energy. For 
microscopic systems, this result must be considered statistically, as fluctu- 
ations around average behavior become substantial. In recent years it has 
become recognized that these fluctuations satisfy a number of strong and 
unexpected relations, which remain valid even when the system is driven 
far from equilibrium. We discuss these relations, and consider what they 
reveal about the second law of thermodynamics and the nature of irre- 
versibility at the microscale. 

1 Introduction 

Looking back at the nineteenth century, we should not be surprised that the 
formulation of thermodynamics coincided with the industrial revolution. Ther- 
modynamics provides a deep and indispensable understanding of the principles 
by which engines and refrigerators operate, and the fundamental limits they 
must obey. The development of thermodynamics in turn gave rise to statistical 
mechanics, as Boltzmann, Maxwell, Gibbs and others sought the link between 
concepts such as heat and temperature, and the Newtonian motion of atoms and 
molecules. From these developments there emerged our modern understanding 



1 



of thermodynamics as an effective theory for macroscopic systems, arising from 
a statistical treatment of their microscopic constituents. The success of this 
approach rests the law of large numbers, which dictates that the greater the 
number of constituents of the system, the smaller the relative size of deviations 
(fluctuations) from the average behaviour. For macroscopic systems, significant 
deviations from the typical behaviour are virtually impossible, hence computed 
averages coincide with measured values of the observables. 

We are now in an era where the microscopic world is increasingly accessible, 
as evidenced by dramatic progress in areas of research such as single-molecule 
manipulation and nanotechnology. Advances in experimental techniques, the- 
oretical modeling and numerical simulation have produced an increasingly de- 
tailed understanding of biomolecular machines such as myosin, kinesin, and the 
complex apparatus responsible for the replication, transcription and transla- 
tion of the genetic code in living cells [HI]. Moreover, laboratories around the 
world are using the techniques of supramolecular chemistry to synthesize molec- 
ular complexes with moving parts [23]. For such microscopic systems thermal 
fluctuations are centrally important, and it is no longer sufficient to focus our 
attention exclusively on average behavior. This, together with natural scientific 
curiosity, has led to increased interest in asking what the laws of thermodynam- 
ics "look like" , when applied to microscopic systems [I] . 

In this text, we address this issue, focusing in particular on the behavior of 
systems driven away from an initial state of thermal equilibrium. We will discuss 
several results that pertain to such systems, and that involve the relationship 
between work and free energy. While these results are in principle valid quite 
generally, in practice they are relevant mostly to microscopic systems, for which 
fluctuations are substantial. 

As this text is a pedagogical discussion, we will start with a brief review of 
the relevant macroscopic thermodynamic relations, using the familiar example 
of a stretched rubber band. We will then consider how these relations might 
apply to a microscopic analogue: a single RNA molecule stretched with opti- 
cal tweezers. Here the laws of thermodynamics, particularly the second law, 
must be interpreted statistically, in terms of averages over many measurements. 
However, it is the fluctuations around these averages that will be the object 
of central interest in this text: as we will describe in detail in Section 21 these 
fluctuations - which we might be tempted to dismiss as mere "noise" - in fact 
satisfy a number of strong and unexpected relations, which remain valid even 
far from thermal equilibrium. 

This text was prepared in large part by two of the authors (E.B. and B.W.), 
based on lectures given by the third author (C.J.) at the International Summer 
School on Fundamental Problems in Statistical Physics XII (FPSPXII), held in 
Leuven, Belgium in September, 2009. It is meant to provide an introduction 
to the topics it covers, in a style that reflects the informal and stimulating 
atmosphere of the summer school. 
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Figure 1: Stretching a rubber band 



2 Brief review of thermodynamic processes 
2.1 An experiment with a rubber band 

Consider the system shown in Figure [TJ One end of a rubber band is attached 
to an immovable wall, the other end to a spring on which a force can be exerted. 
The length of the rubber band will be denoted by z and the combined length 
of the rubber band and spring will be denoted by A. The system interacts with 
the surrounding air (thermal environment) which is at temperature T. The 
parameter A will be controlled externally and will be used to exert work on the 
system. We assume that for every fixed value of A and T there is a unique 
equilibrium state, to which the system relaxes if undisturbed. 

We will use the term thermodynamic process to denote a sequence of events 
during which a system evolves from one equilibrium state to another. During 
such a process, work (W) is performed on the system, heat (Q) flows into the 
system from the environment, and the internal energy of the system may change 
(AC/). The first law then reads 



There is some subtlety even in this seemingly straightforward statement of en- 
ergy balance. Namely, the definitions of W, Q and AU depend on how we define 
our system of interest. For example, in Figured] we could take the system of 
interest to be cither the rubber band itself, or the rubber band together with 
the spring. The corresponding definitions of work are then, respectively, 

• system = rubber band alone : W = J F spr i ng dz 

• system = rubber band + spring : W — J F spimg d\ 

where Spring is the instantaneous tension of the spring. The difference between 
these two values is simply the net change in the energy of the spring itself. 
Either choice is valid, but in this text we will use the second definition. 

The second law of thermodynamics asserts that there exists a state function, 
entropy (S), that obeys the Clausius inequality, 



AU = W + Q 
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for any process that begins with the system in equilibrium state A and ends 
in equilibrium state B. Here the inexact differential dQ denotes an amount of 
heat absorbed from the thermal surroundings, at temperature T. The equality 
is attained if and only if the process is reversible. 

Let us now imagine the following process: initially the rubber band is at 
equilibrium with the surroundings at temperature T and the initial value A = A 
of the work parameter. Next we quickly stretch the band by changing A to a 
new value B. During this process the rubber band will heat up a bit, so once the 
stretching is complete we let it cool down until it is again at equilibrium with 
the surroundings. For such a process, by combining the first and the second law 
we find : 

W = AU 
= AU 
> AU 

Note that T here denotes the (constant) temperature of the bath, rather than 
that of the system itself. In terms of the Helmholtz free energy, F = U — TS, 
we have 

W > AF (1) 

This inequality is the general statement of the second law for processes in which 
the system of interest is in contact with (at most) a single heat bath. Since we 
will never consider the situation of two or more heat baths in this text, equation 
JTJ is the form of the second law that we will use. 

2.2 Cyclic processes 

We will occasionally consider cyclic processes, that is, processes during which 
the value of the external parameter A is the same at the beginning as at the 
end. For the rubber band example, one can imagine that after the stretching 
procedure described above, one brings the band back to its initial state, by 
varying A from B to A and again allowing the system to equilibrate with its 
surroundings. 

For cyclic processes we have AF cyc — (since free energy is a state function) 
thus equation (J} becomes 

W cyc > 

This is Thomson's formulation of the second law. 35 Since the net change in 
the internal energy of the system is also zero during a cyclic process {AU cyc = 0) 
we must also have Q cyc < 0. The opposite case (W cyc < < Q cy c), illustrated 
schematically in Figure [3J is not possible for a cyclic process. If it were, we 
could construct a so-called perpetual motion machine of the second kind, a 
device that would convert thermal fluctuations into work. Thus Thomson's 
formulation is essentially a "no free lunch" statement: we cannot (for example) 
harvest the energy contained in the thermal motions of air molecules, and deliver 
that energy as work, simply by repeatedly stretching and contracting a rubber 
band. (We stress that this conclusion is valid in the presence of a single thermal 
reservoir. When two or more heat baths are present, we can obtain work simply 
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Figure 2: The second law prohibits this situation, for cyclic processes. 



by stretching and contracting of a rubber band, as illustrated beautifully in Rcf. 

HBO 

Now let us split the cyclic process that takes A from A to B and back to A 
into a forward and a reverse process. 

• Forward process : A : A — > B 

• Reverse process : A : B — > A 

More precisely, let us assume the reverse process uses the time-reversed protocol 
for varying the work parameter: A^ = A^!_ t , where r is the duration of either 
process. If we now apply (H} to both the forward and the reverse process we 
find 

W F >AF = F B - F A 
W R >-AF = F A -F B , 

equivalently 

-W R < AF <W F (2) 

Here too, the equalities are attained if and only if the process is reversible. 
Equation (0) immediately implies 

W F + W R > (3) 

which is simply Thomson's formulation of the second law. 



3 How might these results apply to microscopic 
systems? 

3.1 Example: stretching a strand of RNA 

To illustrate how the thermodynamic results of the last section translate to the 
microscopic world, it is again useful to consider a specific example. Figure [3] 
schematically depicts an experimental setup for stretching a single molecule, 
a microscopic analogue of the rubber band discussed in the previous section. 
The system of interest here is a strand of RNA, which is chemically attached 
to a polystyrene bead, held fixed by a micropipette (analogous to the wall in 
the rubber band example). The other end of the RNA is also attached to a 
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bead, which is confined by a laser trap (analogous to the spring in the previous 
example). The whole system is surrounded by water at room temperature, 
providing the thermal environment. By varying the position of the laser trap 
(A) work is performed on the system. For actual experimental implementations 
of such single-molecule manipulation experiments, within the context of the 
theoretical predictions of Section SI see Liphardt et al. [2f5] , Collin et al [5] , or 
(using atomic force microscopy instead of a laser trap) Harris and Song |14j . 




Figure 3: Stretching a single molecule. Note that this figure is not to scale: in 
reality the beads and DNA handles are much larger than the strand of RNA. 

Now imagine the following irreversible process: start at A = A, with the 
molecule in equilibrium with the thermal environment, then quickly stretch the 
molecule by varying A to a new value B, then let the system relax to a new 
equilibrium state, at A = B. Once this sequence of events is complete, change A 
back to A, following the reverse protocol, as in the previous section, and let the 
system relax again to equilibrium. As above, let Wf and Wr denote the work 
performed during the forward (stretching) and reverse (contraction) stages, and 
W cyc — Wf + Wr. If we repeat this cyclic process many times, we will not be 
surprised to measure different values of work, from one repetition to the next, 
as thermal fluctuations of the RNA strand and the surrounding water molecules 
play a significant role at this microscopic scale. We might even encounter rare 
occasions for which W cyc < 0; these are in effect fortuitous events, during which 
the thermal motions of the water molecules happen to facilitate the stretching 
and contraction of the RNA strand. Because fluctuations are not negligible at 
this microscopic scale, we should interpret the second law as a statistical law, 
pertaining to expectation values, rather than a prediction about any particular 
realization of an experiment. Thus equation ([2]) is replaced by the following 
statement about average work values: 

(-W R ) <AF< (W f ) (4) 

Here and throughout this text, angular brackets denote an average over many 
realizations (repetitions) of the process. For the full cycle we have 

(W cyc ) = (W F + W R ) > 

These considerations are illustrated in Figure^ where work distributions pf(W) 
and pr(— W) for the forward and reverse processes are depicted. We see that 
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the average work values satisfy equation but each distribution shows a 
substantial spread of possible work values. Note that the distributions have 
been drawn to intersect precisely at W — AF; this is not a coincidence, but 
rather the consequence of equation (0) discussed in Section H] below. 




Figure 4: Work distributions of the forward and reverse processes. 



3.2 The second law from microscopic arguments 

The discussion in Section 13.11 can be summarized as follows: while the Clau- 
sius inequality might be violated during individual realizations of the process, 
W < AF, it will be satisfied on average, (W) > AF. This conclusion is an 
intuitively natural way to extend the second law to microscopic systems, where 
fluctuations are non- negligible. It is interesting to ask whether we can actu- 
ally justify this intuitive conclusion using microscopic principles. This is a deep 
question that has led to both insight and controversy, and we will not attempt 
to frame a satisfactory answer here. Nevertheless, it is instructive to go through 
some simple analysis of an isolated Hamiltonian system, evolving under a cyclic 
process. (In the discussion below, it is useful to keep in mind that this "isolated 
system" might be a proxy for the combination of a system of interest and a heat 
bath, if we consider the two together as a large, isolated system.) 

Let the microstate x = (q,p) = (q\, . ■ ■ , q m ,Pi . . . ,p m ) describe the positions 
and momenta of all the particles in the system. The 6m-dimcnsional space that 
x belongs to is the phase space T. The Hamiltonian H\(x) is a function of 
this microstate and one or more parameters A, and governs the dynamics by 
Hamilton equations: 

, _ dH , _ dH 

^ dp 1 ^ dq 

This describes a fully deterministic dynamics, i.e. when x(0) is given, x(t) is 
known for any later time t. This x(t) (or xt) defines what we call a trajectory 
or path through the phase space. 

Hamiltonian dynamics satisfy Liouville's theorem: take a subset C of the 
phase space, with volume |C|. If we let all the points x € C evolve under Hamil- 
tonian dynamics to a later time t, this defines a new subset Ct = {x(i)|x(0) G C} 
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with volume \Ct\- Liouville's theorem then states that \Ct\ = |C|, or equiva- 
lently: 

dx t = / dxi{t) \ _ i 
Bxq \dxj(0) J 

For an isolated system, the first law of thermodynamics dictates that 

W = AU = H B (x t ) - H A (x ) 

where again A and B denote the values of the external parameter A at the 
beginning and the end of the evolution. For a cyclic evolution of the parameter, 
A = B. We can now pose the question: does the inequality W cyc > hold 
for every initial condition x(0)? In other words, for this isolated system is 
Thomson's law satisfied for every trajectory? Without attempting a rigorous 
analysis, the following simple argument suggests the answer is "no" . 




Figure 5: Hypothetical phase diagram illustrating the scenario C C C T . 

For simplicity, imagine a Hamiltonian Ha (x) with a single minimum in phase 
space, and consider the set of initial conditions C corresponding to all mi- 
crostates with an energy lower than some value Uq, see Figure [S] (on the left). 
Let C T denote the set of final conditions reached after Hamiltonian evolution 
during a cyclic process. There are three scenarios for the relationship between 
these two subsets of phase space: C % C T , C c C T , and C — C T . The first 
implies that there exists some point x that belongs to C but not to C T . If 
we now consider a trajectory that ends at this point, then by construction the 
initial and final conditions for this trajectory satisfy: 

xq £ C , x T — x G C (5) 

Hence W cyc < for this trajectory. The second scenario (C C C T ) is depicted 
on the right side of figure and implies \C T \ > |C|, a clear contradiction with 
Liouville's theorem; therefore we can rule out this scenario. Finally, the third 
scenario, namely C — C Tl corresponds to the limiting case of a reversible process, 
for which we expect W cyc = 0. Thus we conclude that (unless the process is 
performed reversibly) there will exist initial conditions for which W cyc < 0. 

The above considerations suggest that the second law ought to be interpreted 
statistically: 




dx f A {x ) [H A (x t (x )) - Ha(xq)] > 
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where /a (20) i s a normalized probability distribution that represents the statis- 
tical state of equilibrium at A = A. Whether or not this inequality is satisfied 
depends, of course, on the choice of distribution /a- In the following paragraph 
we show below that the inequality is satisfied for a canonical distribution of 
initial conditions, Ja (x) oc e~ /3HA< ^ x \ It is also straightforward to establish that 
the inequality is satisfied if /a{%) oc 9(Uo — Ha{x)), where Uq is a constant 
and #(•) is the unit step function. (We leave this as an exercise for the reader.) 
Somewhat surprisingly, however, for a microcanonical distribution of initial con- 
ditions, Ja {x) oc S(Uq — Ha(x)), it is possible to construct explicit examples for 
which (W cyc ) < [301 [32]. 

Consider a map M : T — > T : x 1— > y, for which \dy/dx\ = 1 and for which 
the inverse M _1 exists. An example of such a map is Hamiltonian evolution for 
some fixed time (x — xq, y = x T ). Define furthermore a positive constant /3 and 
the work W(x) :— Ha{v{x)) — Ha(x). We compute the following average: 



Za 
Za J 

dy 



1 

1 



dx 
dy 



(6) 



where Za is the usual partition function. From Jensen's inequality (see Ap- 
pendix A for a brief derivation) it follows that 



> e 



so that indeed (W) > 0. A more general version of this derivation can be found 
in e.g. [23 EI]. 

These very simple considerations should not be viewed as first-principles 
derivations of the second law. Rather, they provide some quantitative justifi- 
cation for the intuition stated at the beginning of this section: the second law 
(which in this case takes the form W cyc > 0) will be violated for individual 
trajectories, but will be satisfied for an average over an ensemble of trajectories, 
at least for certain choices for the distribution of initial conditions. 



4 Nonequilibrium work and fluctuation relations 

The central point of Section [3] above is that - for microscopic systems - the 
second law ought to be interpreted statistically, as a statement about averages. 
We now turn to the main focus of this contribution, namely the fluctuations 
around these averages. We will discuss three prediction, given by equations ([7|). 
dU) and © below, that pertain to these fluctuations. After introducing these 
results, we will derive each of them, using three different schemes for modeling 
the microscopic dynamics of a system driven away from equilibrium. In each 
case, we imagine (as with the single-molecule stretching in Section 13.11 above) 
that the system of interest is prepared in equilibrium with a heat bath, then 
driven away from equilibrium by varying the work parameter over a time interval 
< t < r, from A = A to \ T = B. 



9 



1. The first prediction is the nonequilibrium work relation [19 , 

(e-W) = e-? AF (7) 

As before, W is the work performed during a single realization, AF = 
Fb — Fa is the free energy difference between the initial and final equi- 
librium states, (3 = (ksT)^ 1 is the inverse temperature of the heat bath, 
and angular brackets denote an average over an ensemble of realizations 
of the process. In Section 14.11 below, we will derive equation (JTJ) under 
Hamiltonian dynamics, and we will show that this equality immediately 
implies two inequalities related to the second law: 

• (W) > AF 

• Prob(VF <AF-n k B T) < e~ n 
for any real, dimensionless n > 0. 

2. The second prediction, a fluctuation theorem due to Crooks [BJ [T] , 

Pf{W) _ MW-AF) 



Pr(-W) 



will be proven in Section 14.21 for stochastic jump processes. Note that 
equation ([7]) follows directly from equation ©. Moreover, this result 
implies that the work distributions for the forward and reverse processes 
cross one another at W = AF, as depicted in Figure |H 

3. Equations ([7]) and © in principle allow us to determine an equilibrium 
free energy difference from measurements of nonequilibrium work values. 
The third prediction goes further, giving a prescription for constructing 
the entire equilibrium distribution |17l 118] : 

{5{x-x t )e-^) = ^-e^ H ^ (9) 

where Wt is the work done up to time t for a trajectory xt and Z a is 
the partition function at time t = 0. (Equivalent formulations of this 
result appear in Refs. [501 IB]-) We will derive this result for the case of 
continuous time Markov processes. 

Before proceeding to the derivations of these results, some general remarks 
are in order. First, these results remain valid even if the system is driven far 
from equilibrium during the process. They place strong and rather unexpected 
constraints on the work fluctuations: one would not have naturally guessed 
the validity of equations (J7J) - (0 simply by extrapolating from macroscopic, 
thermodynamic intuition. Next, these results reveal that information about 
equilibrium properties (such as AF) is in effect encoded in the fluctuations of the 
system when driven out of equilibrium. Finally, equations © - © are just three 
results among a considerably larger set of predictions related to fluctuations 
in far-from-equilibrium processes. In particular, the early work of Bochkov 
and Kuzovlev [TJ [21 E] includes related predictions, using a somewhat different 
definition of work. Furthermore, while we consider only fluctuations in work, 
there has been extensive investigation of fluctuations in entropy production, 
which satisfy similar relations; see e.g. Refs. [H [TU1 E21 [M 123 HI1 12H1 [HI 133] - 
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4.1 Hamiltonian dynamics 

Here we derive equation ([7]), using Hamiltonian dynamics to model the micro- 
scopic evolution of the system. Specifically, we imagine that after the system 
has been prepared in equilibrium, at A = A, the heat bath is removed and the 
now-isolated system evolves under Hamilton's equations as the work parameter 
is varied from A to B. At the end of the process the system is again brought into 
thermal contact with the heat bath and allowed to reach equilibrium. This sce- 
nario does not realistically describe many physical situations, such as the RNA 
stretching experiment, therefore the derivation of equation ([7]) that we present 
here is not the most general one. However, it is a particularly simple deriva- 
tion, which avoids some of the technical complications that arise when treating 
a system that remains in contact with a heat bath during the entire process; 
see Ref. [22] for a Hamiltonian analysis that explicitly includes the degrees of 
freedom of the system and the heat bath. 

Our system is described by a Hamiltonian H(x,A), where x = (q ,p). Be- 
cause the system is isolated, the work performed as the external parameter is 
varied from Ao = A to A T = B is 

W = H{x T ,B) - H(x ,A) 

Since Hamiltonian dynamics are deterministic, we can view the final conditions 
as a function of the initial conditions, thus writing 

W = H{x Tl B) - H(x ,A) = H(x T (x ), B) - H(x , A) = W{x ) (10) 

The only stochasticity is in the assignment of initial conditions for a given 
realization, therefore to compute the left side of equation we must average 
over the distribution of these conditions: 

{e^ w )= Jdx r(xo,A)e^ w ^ 
Here f eq is the equilibrium distribution, given by the canonical expression 

P\x,\) = ^e^ H ^ (11) 
We now perform the averaging as in the previous section: 

J Za 
Za J 

_ _ —fSAF 

Z A ~ 

where the free energy is given by the familiar expression, 

F(A) = ~lnZ A AF = F(B)-F(A) (12) 

One consequence of ((7|) is that the second law (in the form of the Clausius in- 
equality) appears as an application of Jensen's inequality: since the exponential 
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function is convex, we have that (e - ^) > e~^ w \ and therefore (W) > AF. 
We can also use equation (|7j! to establish a bound on the probability to observe 
work values that "violate" the second law, W < AF, as follows. If p(W) repre- 
sents the probability distribution of work values, then the probability that the 
work is smaller than AF — nksT, where n > 0, is 

/AF-nk B T 
dWp(W) 
-OO 

/AF-nk B T 
dWp(W)e-^ w ~ AF+nkBT ~> 
-oo 

/+oo 
dWp{W)e-f 3w 
-oo 



In other words the probability to observe work values smaller than AF decays 
exponentially, or faster, in the size of the violation. This is consistent with our 
experience that macroscopic violations of the second law are never observed. 

4.2 Stochastic jump processes 

Here we derive equation (|5]) for a discrete-time Markov dynamics. Instead of 
representing the evolution of the system as a continuous trajectory, we imagine 
a chronologically ordered sequence of configurations, 

Xq — > X\ — >• X2 — > ■ ■ ■ — > X T -\ — > X T 

which might represent "snapshots" of the system at equally spaced intervals 
of time. We imagine that the system remains in contact with the heat bath 
throughout this process, therefore the transitions from one state to the next are 
stochastic. 

These dynamics are specified by the transition probability densities P\ (x 
x') y which depend on time through the parameter A, and which satisfy the 
normalization condition 

dx'P x {x -> x') = 1 (13) 



We assume that the process is Markovian, in other words each transition is 
statistically independent of the previous transitions. We also assume that each 
transition satisfies a detailed balance condition 

where the direction of the arrow indicates the transition. We can interpret this 
condition by imagining for a moment that A is held fixed, allowing the system 
to relax to a state of equilibrium represented statistically by the Boltzmann 
distribution. Equation (fH)) then tells us that in this state the net rate of 
transitions from x to x' is the same as that from x' to x: 
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Of course, we are interested in a process during which the value of A is not 
held fixed, and therefore the system does not (in general) remain in equilib- 
rium. To describe this situation, let us imagine that A changes in between each 
transition. We represent this situation schematically as follows: 

Ao = A xq — > x\ — > X2 — > ■ ■ ■ > x T -i — > A r = B 

Following Crooks [6], now establish a link with thermophysics by specifying 
what we mean by heat and work in this model. We define heat as 

T 

Q = ^[H{x t ,X t )-H{x t - 1 ,X t )] (16) 
t=i 

which is the sum of energy changes due to Markov transitions, while the work 
is defined as 

T 

W = J2 [H(x t - U \t) - H(x t - X , A t _i)] (17) 
t=i 

which is the sum of energy changes due to the changing of A. Taking the 
sum of these two contributions, we arrive at a statement of the first law of 
thermodynamics: 

Q + W = H(x T ,B)-H(x ,A) (18) 
Consider now a forward protocol and its reversed protocol, 

A = A£ -»■ Af -»• > Af = B 

A = \? <- Af_! < <- Aj ; = B 

related by Af = A^_ t . Likewise we define a forward and backward trajectory, 
X = (xq -> X\ -> •••->■ x r ) and = (x <- x\ <- ■ ■ ■ <- x T ). If P F (X) de- 
notes the probability density for generating the trajectory X during the forward 
process, and P R (X>) is defined analogously, then their ratio is 

P F (X) fZ'(xo)P xr (xo^X 1 )---P X r(Xr-l^X T ) 
P«(Xt) _ f B \x T )P x n(x T -> X T -l) ■ --P^ixi "> X ) ( ' 

with initial conditions for either process sampled from the corresponding Boltz- 
mann distribution. Combining the assumption of detailed balance (1141) with the 
definition of heat (TlBl) , we find that 



P F (X) fjjxp) 
P«(Xt) f B \x T )' 



-I3Q* 



where Q F is the heat dissipated during the forward path (Q F — ~Q H ). Using 
the explicit expressions for the canonical distributions, we arrive at [6] 

P F {X) _ I3AF+/3AU-PQ F _ 8W F ~I3AF 



P R (X^) 
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where W F — W F (X) is the work done during the forward process. With this 
expression, deriving equation © is simple: 



p F {W) = J dXP F {X)S(W -W F (X)) 

= J dXP R (X^ w ^ x '>- AF '>S(W -W F (X)) 
= e ^ w - AF ) J dX^P R (X^5(W + W R (X^)) 

where dX = dx^dxi ■ ■ ■ dx T — dX^ , and we have used W F (X) — — W R (X^). 

From ((SJ), we see that pp(AF) = pn(-AF), meaning that observing a work 
value equal to the free energy differences is equally (un)likely during either 
process. Furthermore, rewriting equation (j^J as 

p F (W)e-» w = PR (~W)e^ AF 

then integrating over W, we recover equation (J7J- 

4.3 Continuous time Markov dynamics 

Equation @ is essentially an application of the Feynmann-Kac theorem, as 
emphasized by Hummer and Szabo [171 118] . Let us therefore first sketch a (non- 
rigorous) derivation of this theorem in the context of continuous time Markov 
processes. For a more mathematically rigorous approach, see Ge and Jiang |13) . 

Preliminaries: We define P [X\xq, to] to be the probability density to observe 
the trajectory X, given that the system was in state xq at time to. When we 
integrate this over all possible trajectories that end in state x at time t, we get 
the probability density of the system being in state x at time t given that it was 
in state Xq at time tgi 



K{x,t\x Q ,t Q )= J dXP[X\x ,t 



and K(x,to\xo,tQ) = 5(x — xq). (The notation J dX here denotes integration 
over all intermediate states of the trajectory, with the initial and final points 
fixed.) Note that J dxK(x,t \xo,to) should be equal to one. As we are working 
with a Markov process, one can use the Chapman-Kolmogorov relations to write 
a differential equation for K: 

d f 

— K(x,t\x ,t ) = / dx'R t {x' -> x)K(x ,t\x ,t ) 



dt 

where Rt is an instantaneous transition rate, given by 

K(x,t + At\x' : t)-5(x-x>) 

Htlx — >■ x) = lim r 

y ' At-X) At 

and as a consequence J dxRt{x' — > x) — 0. We schematically write the differen- 
tial equation as 

§- t K = L t K (21) 
where the operator L t is called the generator of the Markov process. 
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Feynman-Kac: Now let us consider giving each trajectory a time-dependent 
statistical weight. Specifically, we define an arbitrary function ui(x, t), and the 
trajectory-dependent quantity 

ft(X) := [ uj{x s ,s)ds (22) 

J t 

With this, we define a quantity analogous to K: 

G{x,t\x Q ,t ) := J dXP[X\x ,t ] e° w 

where the difference lies in the exponential weight assigned to each path. The 
goal now is to write down a differential equation for G analogous to (|2"Tj) . For a 
short time interval dt we have 

= e" {xut)dt K{x,t + dt\x',t) + o{dt) 

= e^'^lSix - x') + R t {x' -» x)dt] + o(dt) 



so that 

G(x,t + dt\x ,t ) = J dx'G(x,t + dt\x' ,t)G(x' ,t\x ,t ) 

= (1 + u>(x, t)dt + dtL t )G(x, t\x ,t ) + o(dt) 
This then finally leads us to: 

BG 

-^ = {u + L t )G (23) 

This result is the Feynman-Kac theorem in the present context: when exponen- 
tial weights given by equation (|2"2"|) are assigned to the trajectories of a process 
with transition probabilities K satisfying (12111 . then the weighted transition 
probabilities G satisfy ([23)) . 



Proof of equation ([9]): First note that the probability that the process is in 
state x at time t can be written as 

f(x,t) = (5(x-x t )) 

As this is in our physical systems equal to J dxof^(xo)K(x,t\xo,tQ) for any 
to < t, we see that / itself satisfies 

91 L f 

Recalling that the time-dependence comes from the external work parameter, 
we write L t = L\ t . We further make the assumption of detailed balance, as 
above: 

Lxe -PH(*,\) = 
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for every A. The work done during a particular process, up to time t, is equal to 



By the Feynman-Kac theorem we know the differential equation for G, and 
therefore also the one for g: 



It can be easily checked that g(x, t) oc e -0 H ( x ^ x t) so lves this differential equation, 
and because the initial conditions are g(x, 0) = f^(x), we finally arrive at the 
desired result: 



5 Guessing the direction of the arrow of time 

We conclude with a brief discussion of the nature of irreversibility at the mi- 
croscopic scale, framed in the context of a thought experiment that involves 
guessing the chronology of a sequence of events. 

Suppose you are shown a movie of a tennis ball falling freely in the absence 
of significant frictional forces, for instance near the surface of the moon. If asked 
whether the movie is shown as it actually happened, or whether it is being run 
backward in time, you will not be able to answer; the dynamics of the ball 
is symmetric in time. On the other hand, when you are shown a movie of a 
glass of wine falling and shattering on the ground, or of the reverse - shards of 
broken glass spontaneously assembling themselves and then collecting a stream 
of wine - there is no difficulty determining which version depicts the actual 
sequence of events. Between these two limiting situations, there are cases like 
the RNA pulling experiment, where frictional forces are certainly non- negligible, 
but thermal fluctuations complicate the picture. Let us analyze this situation 
using the results of the previous section. 

As with the examples of the tennis ball and glass of wine, suppose you are 
shown a microscopic "movie" depicting an RNA molecule being stretched, from 
which you are able to reconstruct the trajectory X, as defined earlier. Note 
that the corresponding time- reversed trajectory depicts a realization of the 
time-reversed process, involving the contraction of the RNA molecule. From 
the data - that is, the movie - you are asked to guess the direction of time's 
arrow: does the movie show the sequence of events as they actually occurred 
(e.g. stretching) or are you instead observing a movie that was filmed during the 
reverse process (contraction), now run backward in time? Let F, for "forward", 
and R, for "reverse" , denote these two possibilities, respectively. 

Since both X and X^ represent perfectly valid microscopic trajectories, there 
is no definitive answer to the above question. Rather, the problem becomes an 





g(x,t) = (S(x - x t )e-? w *) = ± e ^ H ^ 
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exercise in likelihood estimation: given the data (X, equivalently X^) which of 
the two hypotheses, F or R, is more likely? The answer is provided by Bayes' 
rule in probability theory: assuming we have no a priori reason to favor one 
hypothesis over the other, the likelihood L(F\X) of the forward hypothesis, 
given the observed data X, is proportional to the probability to generate that 
data when performing the forward process: L(F\X) oc P F (X). The analogous 
statement holds in the reverse case: L(R\X^) oc P R (X j '). Combining these with 
a statement of normalization, namely L(F\X) + L(R\X^) = 1, we conclude that 

L(F\X) = 



P F (X)+P R (Xi) 
We now invoke equation ([20)1 to arrive at the result, 

L ( F \ X ) = 1 + e J(w-AF) ( 24 ) 

This result has previously been derived, with a somewhat different intepretation, 
in the context of free energy estimation 29, 34 . Recall that W is the work done 
during the process X . In Figure[5]we see a graphical depiction of L(F\X). When 



L(F I X) = 



1 



1 + e 



-/3(W-AF) 



L(F\X) 



AF 



W[X] 



Figure 6: A graphical depiction of L(F\X). 

W > AF, it is more likely that we saw the movie in the right order - that is, 
the molecule was actually stretched - while for W < AF, it is more likely 
that we saw the movie in reverse. The transition from L ~ to L ~ 1 occurs 
over an interval of W of width ~ ksT. Thus unless the work is within a few 
UbT of the free energy difference, we can determine the direction of the arrow of 
time with near certainty. This is entirely consistent with our observation that we 
have no problem distinguishing the direction of time's arrow for macroscopically 
irreversible processes like that shattering of a glass of wine. 
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A Jensen's inequality 

Take a space f2 of configurations, and an arbitrary probability density p(x) 
on that space. Denote averages of functions / with respect to p by (/} = 
J dxp(x)f(x). Now consider a convex function f(x). By the definition of con- 
vexity, this function satisfies 

fix) > f(a) + (x- a)f(a) 

for any x, a € f2. Equivalently, we can write 

f(9(x)) > f(a) + (g(x) - a)f'(a) 

for an arbitrary (not necessarily convex) function g(x). Multiplying both sides 
by p(x) and integrating over we get 

(f(g))>f(a) + ((g)-a)f'(a) (25) 

If we now choose a such that a — (g), we arrive at Jensen's inequality: 

(fig)) > f«g)) 

For the special case g(x) = x, this becomes 

</(*)> > /«*» 

References 

[1] G.N. Bochkov and Y.E. Kuzovlcv, Sov Phys JETP 45, 125 (1977) 

[2] G.N. Bochkov and Y.E. Kuzovlcv, Physica 106, 443 (1981) 

[3] G.N. Bochkov and Y.E. Kuzovlcv, Physica 106, 480 (1981) 

[4] C. Bustamante, J. Liphardt and F. Ritort, Physics Today 58, 43 (2005) 

[5] D. Collin et al, Nature 437, 231-234 (2005) 

[6] G.E. Crooks, J Stat Phys 90, 1481-1487 (1998) 

[7] G.E. Crooks, Phys Rev E 60, 2721-2726 (1999) 

[8] G.E. Crooks, Phys Rev E 61, 2361-2366 (2000) 

[9] D.J. Evans, E.G.D. Cohen, and G.P. Morriss, Phys Rev Lett 71, 3616 
(1993) 

[10] D.J. Evans and D.J. Searles, Phys Rev E 50, 1645 (1994) 

[11] R.P. Feynman, R.B. Leighton, and M. Sands, The Feynman Lectures on 
Physics (Addison- Wesley, Reading, MA, 1966), Vol I. Chap. 44-1. 

[12] G. Gallavotti and E.G.D. Cohen, J Stat Phys 80, 931 (1995) 

[13] H. Ge and D.-Q. Jiang, J Stat Phys 131, 675-689 (2008) 



18 



[14] N.C. Harris, Y. Song, C.-H. Kiang, Phys Rev Lett 99, 068101 (2007) 

[15] R.J. Harris and G.M. Schiitz, J Stat Mech: Theor Exp P07020 (2007) 

[16] J. Howard, Mechanics of Motor Proteins and the Cytoskeleton. (Sinauer, 
Sunderland, 2001) 

[17] G. Hummer and A. Szabo, Proc Natl Acad Sci 98, 3658-3661 (2001) 

[18] G. Hummer and A. Szabo, Ace Chem Res 38, 504-513 (2005) 

[19] C. Jarzynski, Phys Rev Lett 78, 2690-2693 (1997) 

[20] C. Jarzynski, Phys Rev E 56, 5018-5035 (1997) 

[21] C. Jarzynski, Phys Rev E 65, 046122 (2002) 

[22] C. Jarzynski, J Stat Mech: Theor Exp P09005 (2004) 

[23] E.R. Kay, D.A. Leigh and F. Zerbetto, Angew Chem Int Ed 46, 72 - 191 
(2007) 

[24] J. Kurchan, J Phys A: Math Gen 31, 3719 (1998) 

[25] J.L. Lebowitz and H. Spohn, J Stat Phys 95, 333 (1999) 

[26] J. Liphardt et al, Science 296, 1832-1835 (2002) 

[27] C. Maes, J Stat Phys 95, 367 (1999) 

[28] C. Maes and K. Netocny, J Stat Phys 110, 269 (2003) 

[29] P. Maragakis et al, J Chem Phys 129, 024102 (2008) 

[30] R. Marathe, J.M.R. Parrondo. larXiv:0909.067lV l 

[31] H. Oberhofer, C. Dellago, P.L. Geissler, J Phys Chem B 109, 6902-6915 
(2005) 

[32] K. Sato, J. Phys. Soc. Japan 71, 1065-1066 (2002) 

[33] E.M. Sevick et al, Annu Rev Phys Chem 59, 603-633 (2008) 

[34] M.R. Shirts et al, Phys Rev Lett 91, 140601 (2003) 

[35] W. Thomson (Lord Kelvin), "On the Dynamical Theory of Heat", Trans- 
actions of the Royal Society of Edinburgh, March, 1851. 



19 



